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Tactical  warfighter  networks  represent  the  final  leg  of  the  Network  Centric  Warfare  space  to 
mud  continuum.  The  challenges  associated  with  developing,  evaluating,  and  fielding  these 
networks  are  significant,  as  experience  from  field  evaluations  demonstrates.  Even  more  critical 
is  the  capability  to  quantitatively  measure  the  extent  to  which  tactical  networks  serve  the 
Warfighters  who  depend  on  them  for  data  and  information.  Such  analysis  is  constrained  in  two 
respects.  The  first  is  the  lack  of  measures  for  the  reliable  correlation  of  human  performance  to 
network  Quality  of  Service  levels.  The  second  is  the  lack  of  applied  data  collection  methodologies 
for  objective  analysis  of  timeliness  and  accuracy  of  decision  inputs  in  the  context  of  integrated 
Command,  Control,  Communications,  Computers,  Intelligence,  Surveillance,  and  Reconnais¬ 
sance  networks.  This  article  reports  on  research  applications  addressing  both  issues.  Our  findings 
were  developed  over  a  5-year  period from  experience  in  a  tactical  networked  operations  field  test 
environment.  We  describe  a  methodology  for  the  reliable  collection  of  human  situational 
awareness  measures  and  report  human  peformance  findings  in  the  context  of  network  metrics. 

We  suggest  emerging  linkages  between  human  and  network  performance  metrics.  Our 
conclusions  recommend  future  actions  that  will  support  user-centric  test  and  evaluation  of 
tactical  networks,  systems,  and  networked  Command  and  Control. 

Key  words:  Decision  accuracy;  decision  timeliness;  human  performance;  mobile  ad  hoc 

4.  These,  in  turn,  dramatically  increase  mission 
effectiveness. 

The  relative  ease  of  analysis  at  the  higher  command 
echelons  (e.g.,  Joint  Operations,  Coalition  Air  Oper¬ 
ations  Centers)  is  determined  in  most  part  from  the 
stable  networks  that  are  utilized  in  these  settings  to 
link  centralized  and  remote  nodes  that  are  stable  in 
terms  of  location  and  satellite  links.  If  viewed  from  the 
lens  of  the  tactical  echelon,  however,  the  four  tenets  of 
NCW  are  less  intuitive,  highly  dependent  on  the 
variable  performance  features  of  Mobile  Ad  Hoc 
Networks  (MANETs),  and  require  adaptive  Com¬ 
mand,  Control,  Communications,  Computers,  Intelli¬ 
gence,  Surveillance,  and  Reconnaissance  (C4ISR) 
human  system  interface  capabilities  that  can  mitigate 
the  MANET  drop-off/self-healing  node  design.  For 
example,  in  his  analysis  of  the  Operation  Iraqi 


network;  situational  awareness;  work  load. 


The  first  decade  of  the  new  millennium 
saw  an  avalanche  of  research  in  Net¬ 
work-Centric  Warfare  (NCW).1  Much 
of  the  early  research  and  lessons 
learned  from  current  operations  fo¬ 
cused  on  the  strategic  and  operational  levels  of 
command  (Conner  2005),  where  the  four  tenets  of 
NCW  (Office  of  Force  Transformation  2005)  were 
somewhat  easier  to  address  compared  with  the  more 
mobile  tactical  force: 

1.  A  robustly  networked  force  improves  information 
sharing. 

2.  Information  sharing  enhances  the  quality  of 
information  and  shared  situational  awareness. 

3.  Shared  situational  awareness  enables  collabora¬ 
tion  and  self-synchronization,  and  enhances 
sustainability  and  speed  of  command. 
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Figure  1.  Network  component  levels  (NRC  2005). 


Freedom  2003  Thunder  Run  (where  lead  elements  of 
the  2nd  Brigade,  3rd  Infantry  Division  attacked 
Baghdad  from  the  southern  outskirts,  through  the  city 
and  west  to  the  airport  [Conner  2005,  p.  15]),  Conner 
noted  that  carrying  the  “robust  intelligence  capability 
[through  a  common  operating  picture]  forward  to  the 
tactical  level  would  prove  almost  completely  lacking” 
(p.  18).  He  characterized  the  existence  of  a  “digital 
divide”  between  operational  and  tactical  commands 
and  suggested  that  the  reasons  for  this  divide  were  the 
great  distances  covered  by  tactical  units  and  the  vast 
amount  of  data  they  were  attempting  to  share.  Conner 
notes  several  examples  in  the  early  phase  of  Operation 
Iraqi  Freedom  where  “the  promise  of  technology 
providing  near  perfect  situational  awareness  had  failed 
the  tactical  commander”  (p.  20). 

Our  central  thesis  is  that  productive  research  in  the 
realm  of  tactical  networking  must  focus  on  linking  the 
physical,  information,  communications,  and  cognitive/ 
social  dimensions  of  the  network.  Intuitively,  this  is  a 
sound  assertion;  human  performance  should  always  be 
investigated  in  the  context  of  supporting  technologies. 
Practically,  this  is  a  large  challenge  in  terms  of 
choosing  metrics  for  comparison  and  in  collecting  data 
for  analysis.  This  point  is  easily  made  by  examining  the 
representation  of  the  network  levels  in  Figure  1 


(National  Research  Council  2005).  If  we  were  to 
choose  one  metric  from  each  level  that  could  be 
expected  to  cluster,  we  might  choose  propagation 
(Physical),  routing  (Communication),  secure  informa¬ 
tion  flows  (Information),  and  shared  understanding 
(Social/Cognitive).  The  first  three  metrics  could  be 
quantitatively  measured,  and  we  could  develop  corre¬ 
lations  among  these  results.  However,  no  quantitative 
metric  is  available  for  shared  understanding  suitable  for 
correlation  with  the  Quality  of  Service  (QoS)  metrics. 
In  addition,  MANET  performance  is  designed  to  be 
dynamic,  with  nodes  dropping  off  and  self-healing  due 
to  terrain  and  weather  conditions.  This  performance  is 
invisible  to  the  human  eye;  users  must  detect  network 
anomalies  from  characteristics  such  as  latency  of 
communications,  failed  messages,  or  garbled  radio 
speech.  Also,  unlike  the  QoS  measures,  human 
performance  measures  such  as  shared  understanding 
will  be  degraded  for  several  reasons;  network  perfor¬ 
mance  is  only  one  contributor.  Experience,  training, 
workload,  and  fatigue  provide  additional  factors  that 
contribute  to  shared  understanding.  Partitioning  out 
the  variance  in  this  factor  due  to  network  performance 
adds  complexity  to  the  human-network  analysis 
problem.  As  we  address  this  issue,  we  briefly  consider 
the  network  environment  that  serves  as  the  setting  for 
our  research. 

U.S.  Army  tactical  networks  will  be  MANETs, 
characterized  by  wireless  radios  with  limited  band¬ 
width  and  no  fixed  infrastructure  support.  Instead  of 
fixed  network  nodes,  the  MANET  nodes  will  be 
dynamic;  they  will  enter  and  leave  the  network  at  any 
time  due  to  mobility  and  terrain  conditions  (Chiang  et 
al.  2008;  Ikeda  et  al.  2009).  In  MANETs,  network 
nodes  will  include  vehicles,  dismounted  soldiers,  and 
unattended  sensors  and  unmanned  vehicles.  The 
challenges  associated  with  developing  information 
and  communication  systems  that  can  operate  on  a 
MANET  are  far  from  trivial.  Porche,  Jamison,  and 
Herbert  (2004)  used  a  high-resolution  simulation  to 
model  variations  of  communication  and  network 
parameters  to  determine  the  impact  on  battle  com¬ 
mand  displays.  For  example,  these  authors  determined 
that  if  sensor  inputs  were  limited  to  50  percent  of  the 
run  time,  and  Common  Operational  Picture  (COP) 
update  rates  were  no  more  than  once  per  minute, 
message  completion  rates  could  be  above  75  percent. 
Notably,  no  human-in-the-loop  testing  was  used  to 
determine  the  effect  of  these  parameters  on  decision 
making.  For  example,  COP  update  rates  of  once  per 
minute  would  be  acceptable  for  a  static  force,  if  that 
force  were  moving;  however,  such  a  rate  would  be 
insufficient  for  navigation  from  the  COP.2  Simulations 
are  useful  in  network  performance  testing  because  of 
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Figure  2.  Schema  for  measuring  cognitive  performance  in  context  of  system  and  network  performance. 


the  unpredictable  and  dynamic  nature  of  MANETs  in 
open  terrain  conditions.  They  have  been  used  for  a 
variety  of  performance  testing,  such  as  packet  latency 
(Anna  and  Bassiouni  2006),  wireless  communication 
protocols  (Gao  and  Boscardin  2006),  and  cross-layer 
routing  (Iannone,  Kabassanov,  and  Fdida  2007).  These 
authors,  respectively,  systematically  varied  traffic  rates 
for  system  load  and  data  packet  sizes  (Anna  and 
Bassiouni),  network  density  and  protocol  performance 
(Gao  and  Boscardin  2006),  and  hop-count  rate  and 
transmission  rate  (Iannone,  Kabassanov,  and  Fdida 
2007).  These  evaluations  provide  a  foundation  for  field 
experimentation  in  that  they  support  the  development 
of  physical  MANET  capabilities  and  document 
simulated  network  performance  bounds. 

Field  experimentation  of  human-in-the-loop  exper¬ 
iments  on  MANETs  is  complex  from  several  perspec¬ 
tives.  First,  the  very  nature  of  MANETs  makes 
controlled  variation  of  key  factors  difficult.  MANET 
structure  and  performance  (e.g.,  connectivity,  node 
links,  bandwidth,  message  completion  rates)  is  dynam¬ 
ic  by  design  and  cannot  be  easily  controlled.  This 
inability  to  systematically  vary  performance  parameters 
presents  a  challenge  to  hardware,  software,  and  human 
evaluation  efforts  (Bowman  and  Kirin  2006;  Porche, 
Jamison,  and  Herbert  2004).  Relating  human  perfor¬ 
mance  to  specific  network  and  system  characteristics  is 
a  second  challenge.  For  example,  consider  the  example 
described  above  from  the  Porche,  Jamison,  and 


Herbert  simulation.  What  quantitative  impact  would 
a  50  percent  sensor  feed,  a  1-minute  COP  update,  and 
a  75  percent  message  completion  rate  have  on 
workload  and  situational  awareness?  Further,  what 
impact  would  those  levels  of  workload  and  Situational 
Awareness  (SA)  have  on  timeliness  and  accuracy  of 
decision  making?  These  questions  imply  a  third 
challenge:  measuring  human  performance  in  objective 
and  quantifiable  ways  with  tools  and  techniques  that 
can  be  applied  across  a  range  of  systems,  experiments, 
and  evaluations.  Partial  solutions  to  these  challenges 
have  been  developed  by  the  authors  over  a  5 -year 
period.  In  that  time  we  have  produced  meaningful 
conclusions  in  the  realm  of  the  cognitive  impact  of 
networked  C4ISR  technologies  on  battle  command 
performance  at  the  tactical  level. 

The  schematic  in  Figure  2  provides  a  high-level  view 
of  our  efforts  to  integrate  performance  metrics  across 
the  network  (Physical  and  Communications),  the 
System  (Information),  and  Cognitive  (Social/Cogni¬ 
tive)  domains.  This  diagram  shows  excerpts  from  our 
data  collection  methodology.  Network  performance 
shows  Message  Completion  Rates  (MCR)  and  latency 
of  messages.  System  performance  shows  numbers  of 
spot  reports  and  free  text  messages  sent  during  the 
mission,  and  shows  a  COP  screen  shot  of  sensor  spot 
reports  (the  yellow  clover  leaves).  Cognitive  perfor¬ 
mance  shows  variable  levels  of  workload  in  terms  of 
performance  satisfaction  and  effort  expended  and  three 
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levels  of  SA  over  time.  This  schematic  shows  data  for 
one  record  run  when  the  network  was  performing  quite 
well  (high  MCR  and  low  latency).  This  high  network 
performance  is  borne  out  by  the  high  number  of  spot 
reports  (over  2,000)  and  free  text  messages  (87), 
substantiated  by  the  COP  screen  shot  (yellow  icons 
fade  out  after  5  minutes).  The  resulting  sensor  icons 
appearing  automatically  on  the  display  explains  the 
spike  in  workload  and  the  initially  high  levels  of  SA, 
followed  by  diminishing  SA  as  the  spot  reports  became 
too  numerous  to  track.  This  example  of  the  impact  of 
network  and  system  performance  on  human  perfor¬ 
mance  provides  a  view  to  our  study  objectives.  In  the 
remaining  sections  of  the  article,  we  delve  into  the 
network  and  cognitive  levels  in  more  detail.  We  do  not 
address  system  performance  in  this  article. 

The  remainder  of  this  article  is  organized  as  follows. 
We  provide  a  brief  description  of  the  venue  in  which 
our  measures  and  methodology  were  developed.  We 
then  illustrate  the  data  collection  methodology  that  has 
proven  useful  for  our  purposes  as  detailed  above.  Next, 
we  review  the  network  and  human  performance 
measures  used  in  our  analysis.  Finally,  we  offer 
conclusions  that  can  extend  our  work  for  test  and 
evaluation  analyses  of  tactical  networks  and  system-of- 
systems  applications. 

Historical  perspective:  C4ISR  On  The 
Move  (OTM) 

Since  2005,  we  have  engaged  resources  at  the  annual 
Communications  Electronics  Research,  Development, 
and  Engineering  Center  (CERDEC)  C4ISR  OTM 
experiments  to  determine  how  tactical  soldiers  will 
benefit  from  an  integrated  sensor  and  communications 
suite  to  improve  timeliness  and  accuracy  of  decision 
making.  At  this  venue,  soldiers  use  instrumented 
vehicles  and  various  mounted  and  dismounted  com¬ 
munications  devices,  interact  with  unmanned  air  and 
ground  vehicles  and  unattended  ground  sensors,  and 
view  Battlespace  entities  on  a  COP  that  is  an  enhanced 
version  of  Force  XXI  Battle  Command  Brigade  and 
Below  (FBCB2).3  Annually,  a  range  of  C4ISR 
technologies  are  integrated  in  a  tactical  network  for 
soldiers  to  use  against  live  Opposition  Forces 
(OPFOR)  (PM  C4ISR  OTM  2005;  PM  C4ISR 
OTM  2006;  PM  C4ISR  OTM  2007;  PM  C4ISR 
OTM  2008;  PM  C4ISR  OTM  2009).  Soldiers 
operated  in  operationally  relevant  missions  daily 
against  a  live,  but  scripted,  enemy  force.  All  enemy 
forces  (vehicles  and  personnel)  were  also  instrumented. 
This  important  capability  is  further  described  in  the 
following  section. 

Our  early  efforts  at  documenting  the  cognitive 
impact  of  an  integrated  and  networked  sensor  suite 


depended  on  a  large  force  of  human  data  collectors  to 
observe  soldiers  at  multiple  node  locations  and  to 
administer  surveys  at  strategic  points  in  the  runs  (PM 
C4ISR  OTM  2005;  PM  C4ISR  OTM  2006;  PM 
C4ISR  OTM  2007).  Although  we  triangulated  our 
data  collection  (Cresswell  1998)  with  multiple  inputs 
(observation,  survey,  individual  and  group  interviews), 
we  determined  that  a  field  study  of  this  type  demanded 
novel  methods  that  were  less  human-intensive  and 
intrusive  into  the  soldiers’  experiences  (Bowman  and 
Kirin  2006).  In  subsequent  years,  we  modified  the 
Army  SALUTE  (Size,  Activity,  Location,  Unit, 
Time,  Equipment)  report  to  include  two  subsequent 
fields:  Assessment  and  Prediction.  The  subsequent 
SALUTE-AP  survey  was  an  effective  tool  to  extract  SA 
reports  from  soldiers,  but  it  still  required  human 
intervention  and  Subject  Matter  Expert  (SME)  scoring 
of  reports  (Bowman  and  Thomas  2008,  2009).  The 
SALUTE-AP  tool  also  restricted  SA  findings  to 
enemy-specific  information;  no  awareness  of  own  forces 
was  included.  And,  while  the  SALUTE-AP  represented 
an  improvement,  it  was  still  a  subjective  report.  We 
continued  to  search  for  objective  measures  that  would 
serve  to  support  the  subjective  reports.  The  Geospatial 
Environment  for  Command  and  Control  Operations 
(GEC20)4  visualization  tool  finally  provided  the 
analysis  medium  for  which  we  had  been  searching. 

Methodology 

GEC20 

The  GEC20  tool  was  designed  originally  as  a 
Rapidly  Operational  Virtual  Reality  (ROVR)  system 
to  provide  a  large-scale  three-dimensional  (3-D) 
immersive  visualization  environment.  The  system 
allows  users  to  interact  with  terrain  models  from 
Google  Earth  as  well  as  high-fidelity  models  created  by 
the  developers.  The  modeled  terrain  provides  extreme¬ 
ly  accurate  representations  of  wooded  and  urban 
training  ranges  at  Fort  Dix  where  missions  were 
conducted.  Map  overlays,  which  can  be  easily  printed 
for  later  use,  are  created  using  standard  drawing  tools 
and  embedded  2525B  military  symbology.  The  ROVR 
supported  both  virtual  walk-through  and  flyover 
navigation  through  its  modeled  terrain.  It  received 
and  stored  all  FBCB2  tactical  messages  and  unit 
position  information  transmitted  over  the  tactical  local 
area  network,  which  was  viewable  in  real  time  or  later 
via  playback.  The  ROVR  was  installed  in  the  Tactical 
Operations  Center  and  configured  using  two  6-  by  4- 
foot  rear  projection  screens  for  an  overall  footprint  of 
about  12-feet  wide  by  7-feet  high  by  8-feet  deep.  The 
soldiers  used  this  capability  infrequently  in  early  years 
due  to  the  remote  location  of  the  large  screen  display. 
Typically,  they  used  rock  drills  as  shown  in  Figure  3. 
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Figure  3.  Rock  drill,  Geospatial  Environment  for  Command  and  Control  Operations  (GEC20)  display,  and  GEC20  mission  planning. 


Also  in  Figure  3  are  the  large  display  and  a  slightly 
smaller  3-D  version  used  for  planning. 

In  2008  and  2009,  the  GEC20  developers  provided 
the  visualization  capability  on  a  smaller  flat  screen  and 
a  tabletop  display  as  shown  in  Figure  4.  The  overlays 
created  in  GEC20  were  electronically  uploaded  into 
FBCB2  in  the  vehicles.  With  these  technologies,  the 
rock  drills  with  sticks  and  pinecones  became  a 
phenomenon  of  the  past. 

A  feature  of  GEC20  that  enabled  it  to  be  used  as  an 
analysis  tool  is  the  ingestion  of  instrumented  position 
reports,  sensor  spot  reports,  and  free  text  messages  for 
geospatial  and  time-stamped  display.  In  2008  and 
2009,  we  used  the  GEC20  to  provide  objective 
analysis  of  subjective  reports  of  SA,  and  discontinued 
the  SALUTE-AP  surveys  (though  we  did  continue  to 
administer  surveys  to  record  workload,  trust  in 
network,  and  usability  concerns  for  technologies).  We 
also  continued  a  short,  four-question  survey  on  SA  in 
order  to  provide  context  to  our  objective  SA  findings. 

Prior  to  the  daily  mission  runs,  the  soldiers  were 
instructed  to  send  SALUTE  reports  through  their 
FBCB2  free  text  messaging  system  whenever  they 
learned  of  new  enemy  activity.  Since  these  were 
captured  and  time-stamped  in  GEC20,  they  provided 
us  an  archive  of  such  reports.  By  viewing  the  map 
display  with  Red  and  Blue  icons  (vehicles  and 


dismounts),  we  could  substantiate  the  SALUTE 
reports  to  determine  accuracy  of  Size,  Location,  and 
Time  elements  of  the  report.  For  Activity,  Unit,  and 
Equipment,  we  relied  on  the  script  for  the  enemy  runs 
and  for  records  of  activity  maintained  by  the  lead 
analyst  assigned  to  the  enemy  force.  We  were  also  able 
to  assess  timeliness  of  reports  through  the  time-stamps 
in  the  GEC20  system.  We  compared  the  message 
received  time  with  the  XML  time  sent  to  calculate 
network  travel  time  for  the  message.  In  addition,  we 
were  able  to  impute  some  system  performance  variables 
in  the  human  decision-making  calculus.  For  example, 
in  GEC20,  sensor  spot  reports  are  displayed  as  yellow 
cloverleaves.  In  Figure  5,  the  display  shows  one  sensor 
field  providing  many  reports  while  another  field 
(marked  by  the  red  circle  in  the  middle  of  the  map) 
provides  none.  Therefore,  enemy  vehicles  approaching 
through  the  inactive  sensor  field  would  arrive  unde¬ 
tected,  whereas  those  arriving  through  the  overactive 
field  could  be  undetected  due  to  the  soldiers’  inability 
to  integrate  a  high  number  of  redundant  reports. 

GEC20  analysis  of  decision-making  accuracy 
and  timeliness 

Timeliness  and  accuracy  of  decision  making  were 
analyzed  with  one  representative  day’s  retrospective 
review  of  data  from  GEC20  playbacks.  This  was  an 


Figure  4.  Geospatial  Environment  for  Command  and  Control  Operations  tabletop  display  and  2-D  flat-screen  display. 
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Figure  5.  Geospatial  Environment  for  Command  and  Control  Operations  screen  shot. 


exploratory  methodology  and  was  not  undertaken  for 
each  day’s  record  runs.  The  run  from  August  20,  2009, 
was  chosen  for  analysis  because  it  provided  a  good 
selection  of  network,  system,  and  human  performance 
data  points.  On  this  date,  12  separate  messages  were 
investigated.  These  messages  are  shown  in  Table  1.  In 
this  table,  we  have  extracted  the  relevant  features  of  the 
message  contents  as  well  as  our  analysis  findings.  The 
messages  extracted  for  analysis  were  all  free  text 
messages  sent  from  one  soldier  (usually  the  Squad 
Leader)  to  the  group  of  soldiers,  all  located  in  stationary 
vehicles.  The  messages  were  generally  uniform  in 
composition  and  size  and  contained  text  that  commu¬ 
nicated  that  soldier’s  knowledge  of  enemy  activity, 
location,  and  size.  The  messages  were  evaluated  for 
timeliness  and  accuracy  according  to  low,  medium,  and 
high  scores.  High  scores  for  these  measures  reflect 
reports  that  contained  activity  descriptions  that  were 
very  close  to  ground  truth  (accuracy)  and  took  little  time 
to  process  (timeliness).  The  time  delays  reported  in  this 
analysis  reflect  network  transit  time,  not  time  delays  due 
to  human  activity. 

The  latency  rates  were  calculated  in  the  following  way. 
Each  message  received  in  the  GEC20  system  showed  a 
“received”  time  and  a  “sent”  time.  The  latter  was  available 
in  the  XML  script  showing  the  message  properties.  The 
analyst  subtracted  the  received  time  from  the  sent  time  to 
calculate  latency.  Review  of  message  latency  times  shows 
that  latency  was  variable  in  the  mission  run;  messages 
were  delayed  significantly  in  the  early  portion  of  the  run 
but  experienced  very  low  latency  throughout  most  of  the 


remainder  of  the  mission.  Latency  began  to  rise  again  at 
the  end  of  the  trial  runs.  This  characterization  of  tactical 
network  performance  shows  the  influence  of  the  airborne 
communications  relay.  On  each  day,  the  airship  had  to 
travel  from  the  nearby  airport  to  the  test  site.  The  early 
and  late  messages  with  high  latency  reflect  message  traffic 
that  used  ground,  rather  than  airborne,  relay  mechanisms. 
Also,  even  when  the  airship  was  present,  it  sometimes 
experienced  problems  that  affected  the  vehicle  network. 
For  example,  if  the  airship  was  not  at  a  high  enough 
altitude  to  affect  the  communications  relay,  ground 
network  pathways  were  used,  resulting  in  network  delays. 

Decision  accuracy 

Decision  accuracy  was  measured  on  a  low-medium- 
high  scale  based  on  the  soldier’s  description  of  enemy 
location  and  activity  compared  with  ground  truth. 
Ground  truth  was  ascertained  from  the  GEC20 
playbacks  of  the  instrumented  OPFOR  vehicles  and 
personnel.  Because  these  position  reports  were  cap¬ 
tured  locally  and  manually  collected  after  each  mission, 
these  reports  are  accurate  on  the  display  to  within 
10  meters.  This  position  accuracy  is  compared  with  the 
BLUE  position  reports  that  are  transmitted  via  the 
satellite  network  and  are  accurate  to  within  20  meters. 
The  BLUE  reports  noted  in  Table  1  contain  informa¬ 
tion  that  the  soldiers  extracted  from  various  sensors, 
including  soldier  visual  detections,  unmanned  aircraft 
system  (UAS)  imagery,  and  simulated  Unattended 
Ground  Sensor  (UGS)  sensor  reports.  The  UAS  and 
UGS  reports  included  a  10-digit  grid  location  of  the 
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Table  7. 

Decision  accuracy  and  timeliness  measures  August  20,  2009. 

Time 
of  report 

Time 
delay  (s) 

Description  of  report 

SME  interpretation  of  report 

Score 

Accuracy  Timeliness 

1029 

349 

Reported  4  vehicles  with  grid  location 

Grid  location  within  14  meters 
of  enemy  vehicle 

High 

High 

1040 

131 

Reported  4  moving  vehicles 

Only  1  was  mobile,  3  were 
stationary 

Medium 

Medium 

1045 

103 

1  Vehicle  reported  at  grid  location 

Grid  location  was  within 

36  meters  of  vehicle 

High 

Medium 

1117 

20 

Reported  11  dismounts  at  grid  location  (based 
on  simulated  spot  report  with  grid  ID) 

Grid  location  was  exact 

High 

Low 

1125 

17 

Reported  1  vehicle  and  1  dismount  (based  on 
simulated  spot  report  with  grid  ID) 

Grid  location  was  exact 

High 

Low 

1129 

21 

Reported  1  stationary  vehicle  (based  on 

UUGS  spot  report) 

Grid  location  was  exact 

High 

Low 

1135 

19 

Reported  1  stationary  vehicle  (based  on 

UUGS  spot  report) 

Grid  location  was  exact 

High 

Low 

1146 

23 

Reported  1  dismount  walking  south  from 
grid  location  (based  on  UAS  image) 

Grid  location  was  within 

10.57  meters 

High 

Low 

1151 

21 

Reported  1  vehicle  driving  (based  on 

UUGS  spot  report) 

Grid  location  was  exact 

High 

Low 

1201 

21 

Reported  1  stationary  yellow  sedan  with  grid 
location  (based  on  DCGS-A  image) 

Grid  location  was  exact 

High 

Low 

1214 

20 

Reported  1  stationary  vehicle  and  1  dismount 

Grid  location  was  exact 

High 

Low 

1215 

80 

Call  for  fire  on  yellow  sedan  (based  on 
simulated  spot  report  with  grid  ID) 

Grid  location  was  exact 

High 

Medium 

SME,  subject  matter  expert;  ID,  identification;  UUGS,  urban  unattended  ground  sensors;  UAS,  unmanned  aircraft  system. 


target.  All  of  the  UAS  sensor  reports  and  some  UGS 
sensor  reports  included  a  simulated  image  (see  Figure  6 
for  examples  of  simulated  images). 

As  might  be  expected,  the  soldier  messages  of  enemy 
activity  that  were  based  on  the  simulated  sensor  reports 
were  highly  accurate.  It  is  clear  why  the  soldiers  were 
so  accurate  using  simulated  sensors;  the  images  are  very 
clear,  and  the  reports  contain  10-digit  grid  locations  of 
the  OPFOR  (e.g.,  in  Figure  6,  the  image  of  the 
burning  vehicle  shows  a  grid  location  of  18  S  WK 
50105  24717). 

Decision  timeliness 

Decision  timeliness  was  also  measured  on  a  low- 
medium-high  scale,  with  low  ratings  being  the  more 
preferred  state.  The  timeliness  ratings  for  each  message 


are  shown  in  the  second  column  of  Table  1.  These 
results  show  a  consistent  trend  in  the  message 
timeliness  factor  noticed  by  experiment  observers  over 
the  3  weeks  of  the  experiment.  The  initial  report 
required  349  seconds  to  reach  the  GEC20  system;  this 
was  followed  by  rapidly  descending  time  periods  until 
the  last  message,  which  began  to  increase  in  time 
required  to  transit  the  network.  The  curve  of  message 
latency  is  shown  in  Figure  7. 

It  is  at  this  point  that  the  difficulty  in  analyzing 
cross-domain  measures  comes  into  play.  When  we 
sought  to  understand  the  reasons  behind  message 
latency,  we  learned  that  the  network  engineers  had  a 
different  perspective  on  that  network  performance 
metric.  The  latency  measures  recorded  by  the  network 
engineers  are  presented  in  Figure  8.  Comparison  of  the 


Figure  6.  Simulated  sensor  images  (Unattended  Ground  Sensor  on  left  and  a  Class  IV  UAS  on  right). 
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Figure  1.  Visual  display  of  message  latency  in  seconds. 


latency  metrics  shows  a  best  (e.g.,  lowest)  latency  of 
19  seconds  in  Figure  7,  but  a  best  latency  of  less  than 
1  second  in  Figure  8.  We  believe  that  this  represents  a 
difference  between  network  level  and  application  level 
measurement;  however,  this  was  not  confirmed  by  the 
network  engineers. 

Figure  8  shows  the  average  latency  of  messages  as 
recorded  by  the  automated  network  data  collection 
tools.  The  one  consistency  between  Figures  7  and  8  and 
Table  1  is  the  spike  at  14:52:48  shown  in  Figure  8  and 
the  349  second  latency  in  Table  1,  row  1.  The  network 
times  were  recorded  in  Greenwich  Mean  Time 
(GMT),  which  was  a  4-hour  additive  time  from  the 
GEC20  recording  format  (Eastern  Daylight  Time).  It 
appears  that  the  network  latency  recorded  a  large  spike 
at  approximately  the  same  time  period  as  the  message 
latency  in  the  application  layer  of  GEC20.  Though 
these  latency  figures  are  far  from  equal  (349  seconds  vs. 
36  seconds),  the  same  phenomena  appear  to  affect  both 


Figure  9.  Future  combat  systems  (FCS)  multi-tiered  transport 
architecture. 


systems.  We  believe  that  the  inverted  bell-shape  curve, 
displayed  in  Figure  7,  is  the  result  of  several  confound¬ 
ing  issues.  Some  of  the  issues  include  network  delays  at 
the  beginning  and  end  of  the  record  runs,  message¬ 
processing  delays  on  the  GEC20  system  because  of 
volumes  of  simulated  position  reports,  a  delay  in 
message  arrivals  due  to  GEC20  system  re-booting 
during  a  mission,  and  differences  in  time  synchroni¬ 
zation  between  the  GEC20  system  time  and  the 
network  time.  Those  differences  are  under  investiga¬ 
tion  by  a  joint  team  of  analysts  from  CERDEC 
Command  and  Control  Directorate,  PM  C4ISR 
OTM  and  the  Army  Research  Laboratory,  and  serve 
to  point  out  the  difficulties  in  this  type  and  level  of 
analysis.  We  will  now  shift  our  discussion  to  cover  the 
cognitive  and  network  measures  used  in  our  analysis. 

Measures:  cognitive  and 
network  performance 

The  emphasis  on  investigating  the  cognitive  impact 
of  an  integrated  C4ISR  suite  of  technologies  was 
driven  by  the  understanding  that  warfare  will  always  be 
a  human  activity.  Technology  must  support  human 
decision  making,  but  it  will  never  replace  it.  The  drive 
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Figure  10.  2013  Modular  brigade  combat  team  architecture. 


to  develop  sophisticated  decision  support  systems  and 
intuitive  visual  displays  must  include  consideration  for 
the  ways  in  which  soldiers  will  utilize  these  systems  on 
the  battlefield.  Networked  C2  places  the  human 
Warfighter  at  the  center  of  a  complex,  dynamic,  and 
uncertain  web  of  information.  This  study  measured  the 
impact  of  networked  human  and  sensor  information  on 
the  cognitive  performance  of  soldiers  at  the  tactical 
level.  This  study  provided  a  benchmark  for  future 
analysis  of  how  valid  the  basic  network  centric  warfare 
tenets  may  be  for  the  tactical  Warfighters.  In  the  next 
section,  we  briefly  introduce  the  network  architecture 
that  supported  these  capabilities  and  review  network 
performance  measures.  We  follow  this  discussion  with 
cognitive  measures. 

Network  architecture  design 

The  architecture  design  elements  of  the  2009  study 
were  drawn  from  several  including  the  Future  Combat 
Systems  (FCS)  Multi-Tiered  Transport  Architecture, 
the  Unified  Battle  Command  120  Day  study  (Moore 
2008)  and  the  2013  Modular  Brigade  Combat  Team 
(MBCT)  Architecture  (Latham  2008).  Each  of  the 
communications  architectures  employed  a  variety  of 
systems  depending  upon  the  specific  configuration 
under  examination,  as  shown  in  Figures  9  and  10. 

Within  these  architectures,  a  common  element 
includes  satellite  communications  provided  by  the 
NCW,  which  support  an  on-the-move  satellite  capa¬ 
bility.  NCW  is  a  communications  waveform  that  acts  as 
the  primary  satellite  mechanism  for  Increments  2  and  3, 


as  well  as  being  available  for  technology  insertion  at 
Increment  1.  NCW  is  an  Internet  Protocol  (IP)-based, 
bandwidth-on-demand  protocol  that  supports  block  file 
data,  voice,  and  video  services,  as  well  as  other  IP-based 
services  to  support  disadvantaged  terminals  that  may 
have  smaller  dish  sizes,  rendering  them  less  capable.  In 
addition,  Soldier  Radio  Waveform  (SRW)  is  common 
to  both  architectures.  SRW  is  an  IP-based,  software- 
defined  digital  communications  waveform  that  supports 
voice,  data,  and  video  services.  SRW  is  designed  to 
provide  communications  to  dismounted  soldiers,  unat¬ 
tended  ground  sensors,  intelligent  munitions  systems, 
non-line-of-sight  launch  systems,  and  unattended 
ground  and  air  vehicles. 

Network  performance 

Several  tables  are  presented  to  illustrate  the  perfor¬ 
mance  of  the  network  during  the  C4ISR  OTM  2013 
MBCT  platoon  trial  runs.  Table  2  represents  the  view  of 
network  performance  from  the  Brigade  Headquarters’ 
perspective.  The  tables  were  derived  from  the  daily  data 
sets  that  were  harvested  after  the  completion  of  the  runs. 
These  data  include  any  unicast  data  between  C4ISR 
Information  Management  System  (CIMS)  and 
FBCB2s,  as  well  as  any  multicast  data  that  originated 
either  from  FBCB2  or  CIMS.  These  multicast  groups 
include  the  standard  groups  for  CIMS  Gateway 
messages,  which  include  imagery  notifications,  imagery 
request,  imagery  transfers,  and  chat. 

The  primary  focus  of  our  attention  in  these  network 
performance  metrics  is  the  lowest  tactical  leader — the 
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Table  2.  Network  performance  -  Brigade  Headquarters  view. 

August  14,  2009  August  18,  2009  August  20,  2009  August  21,  2009 

1337-1653  1311-1757  1313-1630  1717-2000 


From  the  Completion  Average  Completion  Average  Completion  Average  Completion  Average 


Radios 

BDE  HQ_to 

rate 

latency 

rate 

latency 

rate 

latency 

rate 

latency 

NCW 

CO  CDR 

76.9% 

0.60  s 

77.5% 

0.45  s 

95.8% 

0.60  s 

89.4% 

0.59  s 

SRW  CO  Net 

VPL1 

12.7% 

0.95  s 

69.1% 

1.31  s 

87.3% 

1.80  s 

88.0% 

1.32  s 

SRW  CO  Net 

VPL2 

12.5% 

0.95  s 

69.1% 

1.13  s 

87.1% 

1.80  s 

89.2% 

1.33  s 

SRW  CO  Net 

PL 

66.3% 

0.78  s 

58.2% 

0.73  s 

76.9% 

0.83  s 

82.0% 

0.85  s 

SRW  PLT  Net 

SL 

65.3% 

1.00  s 

55.6% 

0.96  s 

75.4% 

1.00  s 

76.8% 

1.00  s 

BDE,  brigade;  HC^  headquarters;  NCW,  network-centric  warfare;  CO,  company,  CDR,  commander;  s,  seconds;  SRW,  soldier  radio  waveform; 
VPL,  virtual  platoon  leader;  PL,  platoon  leader;  PLT,  platoon;  SL,  squad  leader. 


Platoon  Leader  (PL).  These  performance  metrics 
presented  in  Table  2  show  that  the  MCRs  for  messages 
coming  from  the  Brigade  Headquarters  to  the  PL 
ranged  from  58.2  percent  to  82  percent,  with  latency 
measures  of  less  than  1  second. 

The  network  performance  metrics  from  the  Com¬ 
pany  Commander  to  the  PL  on  the  same  dates 
(Table  3)  show  better  MCRs;  the  range  of  MCR  was 
60.7  percent  to  97.1  percent.  We  note  that  only  1  day 
had  a  lower  than  87  percent  MCR,  however.  Again, 
latency  was  a  non-issue,  with  scores  in  the  less  than  1/ 
4-second  range. 

The  PL  perspective,  shown  in  Table  4 ,  shows  fairly 
positive  network  metrics  from  the  MCR  and  the 
latency  categories.  On  August  14,  2009,  the  PL  had 
low  MCRs  with  the  Virtual  Platoon  Leaders  1  and  2, 
but  these  scores  rose  on  the  remaining  days.  The 
completion  rates  from  the  PL  to  his  Company  and 
Brigade  Headquarters  ranged  from  the  mid-80s  to 
mid-90s  range,  with  low  latency  in  all  cases. 

A  comparison  of  the  network  performance  for  these 
seven  trials  offers  a  mix  of  both  expected  and  some 
unexpected  results.  The  network,  composed  of  NCW 
and  SRW,  yielded  a  completion  rate  of  74.8  percent. 
This  result  indicates  that  just  under  three  quarters  of 


the  messages  sent  during  the  trials  actually  reached 
their  intended  recipient,  yet  the  soldiers  were  generally 
able  to  complete  their  assigned  missions.  These  results 
do  not  suggest  that  a  75  percent  message  completion 
rate  is  acceptable  but  rather  illustrates  how  adaptive  the 
soldiers  were  in  accomplishing  their  missions  despite 
network  performance  issues.  For  example,  when  the 
soldiers  were  not  receiving  the  reports  or  messages  that 
they  were  expecting,  they  would  use  the  provided  voice 
communications,  either  Single  Channel  Ground  and 
Airborne  Radio  System  (SINCGARS)  or  SRW,  to 
send  or  request  the  needed  information.  If  they  were 
unable  to  reach  the  intended  person  on  the  radio,  they 
would  frequently  rely  on  personal  cell  phones  to  send 
text  messages  and  images  of  the  OPFOR.  Finally, 
when  other  means  of  communication  were  unavailable, 
the  soldiers  would  simply  drive  to  the  other  person’s 
location  and  conduct  a  face-to-face  meeting.  These 
results  do  not  imply  that  “alternate”  means  of 
communication  were  preferred  by  the  assessment 
team;  rather  they  demonstrate  the  ingenuity  of  the 
participating  soldiers  to  work  around  the  limitations  of 
the  experimental  network.  The  next  section  will 
describe  the  impact  of  the  technologies  on  soldier 
performance. 


Table  3.  Network  performance  -  Company  Commander  view. 


Radios 

From  the 
CO  CDR  to 

August  14,  2009 
1337-1653 

August  18,  2009 
1311-1757 

August  20,  2009 
1313-1630 

August  21,  2009 
1717-2000 

Completion 

rate 

Average 

latency 

Completion 

rate 

Average 

latency 

Completion 

rate 

Average 

latency 

Completion 

rate 

Average 

latency 

NCW 

BDE 

90.0% 

0.74  s 

76.7% 

0.72  s 

75.9% 

0.60  s 

67.8% 

1.37  s 

SRW  CO  Net 

VPL1 

35.4% 

0.44  s 

71.6% 

0.39  s 

78.1% 

0.76  s 

88.4% 

0.65  s 

SRW  CO  Net 

VPL2 

36.3% 

0.44  s 

70.6% 

0.32  s 

77.7% 

0.80  s 

88.4% 

0.65  s 

SRW  CO  Net 

PL 

97.1% 

0.18  s 

87.0% 

0.18  s 

60.7% 

0.21  s 

94.2% 

0.21  s 

SRW  PLT  Net 

SL 

95.2% 

0.39  s 

78.3% 

0.38  s 

59.5% 

0.44  s 

90.7% 

0.42  s 

CO,  company;  CDR,  commander;  NCW,  network-centric  warfare;  BDE,  brigade;  s,  second;  SRW,  soldier  radio  waveform;  VPL,  virtual  platoon 
leader;  PL,  platoon  leader;  PLT,  platoon;  SL,  squad  leader. 
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Table  4.  Network  performance 

-  Platoon  Leader  view. 

August  14,  2009 
1337-1653 

August  18,  2009 
1311-1757 

August  20,  2009 
1313-1630 

August  21,  2009 
1717-2000 

Radios 

From  the 
PL  to 

Completion 

rate 

Average 

latency 

Completion 

rate 

Average 

latency 

Completion 

rate 

Average 

latency 

Completion  Average 
rate  latency 

NCW 

BDE  HQ. 

81.0% 

0.92  s 

69.5% 

1.09  s 

83.4% 

0.87  s 

77.9%  1.50  s 

SRW  CO  Net 

CO  CDR 

85.2% 

0.16  s 

96.7% 

0.22  s 

82.4% 

0.32  s 

96.2%  0.24  s 

SRW  CO  Net 

VPL1 

22.6% 

0.32  s 

72.4% 

0.26  s 

88.8% 

0.91  s 

82.4%  0.55  s 

SRW  CO  Net 

VPL2 

20.6% 

0.34  s 

71.0% 

0.29  s 

87.7% 

0.98  s 

84.4%  0.55  s 

SRW  PLT  Net 

SL 

98.2% 

0.23  s 

69.8% 

0.22  s 

96.9% 

0.22  s 

92.1%  0.23  s 

PL,  platoon  leader;  NCW,  network-centric  warfare;  BDE,  brigade;  HQ^  headquarters;  : 
CDR,  commander;  VPL,  virtual  platoon  leader;  PLT,  platoon;  SL,  squad  leader. 

s,  second;  SRW, 

soldier  radio  waveform;  CO,  company, 

Table  5.  Confidence  in  computer  use. 

Survey  question 

Mean 

Standard  deviation 

Minimum 

Maximum 

I  am  confident  in  my  ability  to  use  computers  in  general. 

4.8 

.41 

4.0 

5.0 

I  am  confident  in  my  ability  to  use  Army  C2  digital  systems. 

4.0 

.89 

3.0 

5.0 

I  am  confident  in  my  ability  to  learn  to  use  new  software  quickly. 

4.5 

.55 

4.0 

5.0 

I  am  confident  in  my  ability  to  perform  multiple  tasks  at  the  same  time. 

4.7 

.52 

4.0 

5.0 

I  am  confident  in  my  ability  to  use  personally  owned  computers. 

4.8 

.41 

4.0 

5.0 

Cognitive  performance  measures 
Demographics 

Soldier  participants  in  this  study  included  five 
Noncommissioned  Officers  (NCOs)  from  the  New 
Jersey  Army  National  Guard  (NJ  ARNG)  and  one 
Officer  from  CERDEC.  All  study  participants  were 
male.  These  soldiers  were  assigned  roles  as  the 
Company  Commander,  Platoon  Leader,  Platoon 
Sergeant,  Squad  Leader,  and  Robotics  operators.  The 
NJ  ARNG  soldiers  had  returned  from  a  year-long 
combat  deployment  earlier  in  the  summer.  The  rank  of 
the  soldiers  was  Major  (1),  Sergeant  First  Class  (1), 
Staff  Sergeant  (2),  and  Sergeant  (2).  The  soldiers 
represented  a  range  of  experience  in  the  military.  Due 
to  the  small  number  of  participants,  experience  levels 
were  divided  into  less  than  and  more  than  10  years  of 
service. 

We  documented  the  soldiers’  subjective  perception 
of  their  ability  to  use  computer  systems  because  the 
Unified  Battle  Command  study  revolved  around  digital 
battle  command  displays.  Table  5  displays  these  data. 
The  questions  were  measured  on  a  five-point  Likert 
scale  from  1  (strongly  disagree)  to  5  (strongly  agree). 
On  average,  the  soldiers  were  extremely  confident  in 
their  ability  to  use  computers  in  general,  to  use 
personal  computers,  to  perform  multiple  tasks  at  the 
same  time,  and  to  learn  new  software  quickly.  Slightly 
lower  scores  were  recorded  for  the  confidence  in  using 
Army  C2  digital  systems.  The  mean  score  of  4.0 
(standard  deviation  [SD]  =  .89)  represented  two 


soldiers  who  rated  this  question  as  “neutral,”  two 
who  “agreed,”  and  two  who  “strongly  agreed.” 

Workload 

Within  the  network  configuration  described  above, 
four  record  runs  were  achieved  from  August  18-21, 
2009.  A  repeated  measure  Multivariate  Analysis  of 
Variance  (MANOVA)  was  used  to  analyze  these  data. 
Though  some  minor  differences  in  workload  were 
noted  in  the  two  experience  groups,  none  of  these 
differences  are  significant  at  the  P  =  .05  level.  As  can 
be  seen  in  Figure  11,  the  less  experienced  soldiers 
consistently  reported  slightly  higher  workload  levels 
than  did  their  more  experienced  counterparts.  How¬ 
ever,  the  highest  level  of  workload  reported  was 
achieved  on  day  four  of  this  MBCT  trial,  and  that 
mean  score  was  38.04,  SD  =  10.36.  Considering  the 
scale  of  the  workload  measure  (0-100),  this  was  a  low 
workload  rating. 

Situation  awareness 

A  repeated  measures  MANOVA  was  used  to 
analyze  these  data,  which  are  graphically  displayed  in 
Figure  12  by  day.  On  each  day,  the  less  experienced 
soldiers  reported  that  they  experienced  more  difficulty 
than  their  more  experienced  counterparts  in  achieving 
Levels  1,  2,  and  3  SA,  but  these  differences  were  not 
significant.  It  should  also  be  noted  that  this  amount  of 
difficulty,  though  relatively  higher  than  the  more 
experienced  soldiers,  is  still  rather  low.  That  is,  none  of 
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Estimated  Marginal  Means  of  Workload 


Figure  1 7.  Average  workload  by  experience  level  in  modular 
brigade  combat  team. 

the  soldiers  reported  much  difficulty  in  achieving  SA  at 
all  levels. 

Conclusions 

This  report  documents  a  number  of  major  contri¬ 
butions  to  tactical  networked  battle  command  engi¬ 
neering  and  soldier  performance  challenges.  The  vision 
of  Unified  Battle  Command  provides  an  evolutionary 
path  for  the  integration  of  dissimilar  applications, 
communication  and  sensor  technologies,  and  forces  in 
full  spectrum  operations.  The  C4ISR  OTM  Unified 

Estimated  Marginal  Means  of  SA 


Battle  Command  Cognitive  Impact  Study  (UBC-CIS) 
explored  the  major  tenets  of  this  vision.  The  findings 
enumerated  in  this  report  validate  the  fundamental 
assumptions  of  UBC  and  clarify  technology  and 
human  challenges  in  realizing  networked  command 
and  control  at  the  lowest  tactical  echelon. 

The  network  architecture  developed  for  the  UBC- 
CIS  provides  a  firm  foundation  for  continued  test  and 
evaluation  of  networked  communication  and  sensor 
technologies  for  tactical  elements.  Specifically,  the 
2009  architecture  confirmed  that  multiple  pathways 
(air-satellite-ground)  for  data  transmission  are  feasible 
with  quantifiable  QoS  consequences  for  each  route. 
These  QoS  impacts  (latency,  message  completion 
rates)  were  correlated  with  human  situational  aware¬ 
ness  and  decision  making.  Knowledge  of  lesser 
collaboration  pathway  consequences  can  be  used  by 
commanders  to  select,  in  advance,  pathways  based  on 
mission  requirements  and  technology  availability. 

This  study  also  examined  technology  and  human 
performance  factors  to  investigate  the  feasibility  of 
extending  the  network  to  the  lower  tactical  elements. 
From  a  technology  perspective,  challenges  were 
documented  in  maintaining  connectivity  with  mobile 
and  static  vehicle  configurations  in  forested  and  open 
terrain.  Soldiers  derived  adaptive  ways  of  sharing 
information  with  less  advantaged  members  of  the  unit. 
Redundant  communication  modes  were  essential  to 
this  capability. 

Estimated  Marginal  Means  of  SA 


summary  experience 

- under  10  year* 

- over  1 C  years 


Estimated  Marginal  Means  of  SA 


Estimated  Marginal  Means  of  SA 


Figure  12.  Reported  levels  of  situational  awareness  of  4  days  of  trials. 
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The  nature  of  networked  communication  and  sensor 
technologies  required  the  tactical  leader  to  be  aware  of 
the  unique  contribution  of  each  system  to  the  mission. 
He  also  needed  to  understand  the  dependencies 
between  systems  and  between  those  systems  and  the 
network.  Given  the  combination  of  air  and  ground 
sensors  and  communication  types,  this  was  not  an 
insignificant  task.  This  experiment  clearly  demonstrat¬ 
ed  the  need  for  the  tactical  leader  in  a  networked  unit 
to  manage  the  network  architecture  for  optimal  system 
and  soldier  performance.  Such  a  requirement  cannot  be 
managed  by  static  tactics,  techniques,  and  procedures 
(TTPs)  alone  given  the  dynamic  nature  of  system  and 
network  performance  in  the  context  of  terrain  and  unit 
configurations.  For  soldiers  to  gain  maximum  benefit 
from  networked  technologies,  these  systems  need  to  be 
deployed  in  an  optimal  configuration  in  the  context  of 
the  mission.  The  role  of  Network  Manager  NCO  would 
provide  services  to  the  force  such  as  field  interpretation 
of  system  specifications  for  optimal  use.  For  example, 
unmanned  and  unattended  sensors  have  unique 
requirements  for  network  support.  When  planning 
for  placement  of  unattended  ground  sensors  (UGS)  or 
routes  for  UAS/UGVs,  network  connectivity  is  a 
primary  factor.  Also,  perception  of  network  health  is 
critical  for  manned  and  unmanned  teaming  and 
selection  of  communication  mode.  For  example,  in 
low  bandwidth  conditions,  text  communications  may 
be  favored  over  voice  or  sharing  imagery.  System 
integration  features  are  also  a  necessary  consideration 
for  successful  network-enabled  performance  at  the 
tactical  level.  Questions  a  Network  NCO  might 
consider  include:  How  are  systems  connected?  What 
are  individual  and  composite  system  capabilities  and 
limitations?  How  does  the  use  of  one  system  impact 
another  system?  What  is  the  link  status  of  network 
nodes?  How  is  node  drop-off  or  re-connectivity 
signaled?  How  can  network  problems  be  diagnosed 
and  repaired?  How  can  errors  be  diagnosed  as  human 
error  (and  subject  to  training  solutions)  or  technology 
failures?  These  issues  will  be  the  subject  of  a  future 
research  effort  as  we  attempt  to  harness  network 
architectures  and  performance  for  tactical  soldiers’ 
benefits. 

The  UBC  vision  recognizes  a  need  for  dissimilar 
units  to  interoperate  with  organic  battle  command 
systems  and  share  common  geospatial  data.  The  UBC- 
CIS  study  examined  the  ability  of  soldiers  to  interact 
with  three  different  battle  command  systems  for  ISR 
missions  against  an  adaptive  and  resilient  enemy  force. 
Though  the  analysis  procedures  were  limited  by  the 
small  number  of  subjects,  this  investigation  provides  a 
good  insight  into  soldier  performance  results  and 
analysis  methodologies  for  exploring  the  cognitive 


impact  of  new  technologies  in  a  mobile  ad  hoc  network 
environment.  In  addition,  the  advanced  capabilities 
demonstrated  by  the  M&S  team  allowed  the  live 
participants  to  interact  with  data  simulating  two 
Brigade  Combat  Teams,  thus  expanding  the  tactical 
perspective. 

The  workload  measures  document  that  the  soldiers 
generally  had  low  workload  scores.  Though  the  less 
experienced  soldiers  did  report  slightly  higher  difficulty 
with  these  tasks,  they  reported,  on  average,  that  this 
was  not  a  particularly  demanding  task.  This  is  due,  in 
large  part,  to  the  effective  use  of  unmanned  technol¬ 
ogies  provided  to  the  soldiers  for  this  experiment.  The 
coordinated  use  of  air  and  ground  vehicles  and 
unattended  ground  sensors  allowed  the  soldiers  to 
maintain  an  awareness  of  enemy  activity  in  the  area  of 
operations.  This  integration  of  manned  and  unmanned 
teaming  for  ISR  highlights  the  major  contribution  of 
the  Cognitive  Impact  Study. 

This  study  represents  the  authors’  efforts  to  plan, 
execute,  and  analyze  complex  instrumented  and  human 
performance  data  in  the  context  of  the  system-of- 
systems  demonstrated  in  this  experiment:  Network, 
Systems,  and  Cognitive.  This  analysis  capability  was 
made  possible  by  the  focused  contributions  of  the 
entire  PM  C4ISR  OTM  experimentation  and  data 
collection  staff,  and  required  a  complex  set  of 
instrumentation  capabilities,  data  reduction  methodol¬ 
ogies,  human  data  collection  staff  (to  observe  and 
administer  human  performance  surveys),  and  senior 
researchers  on  site  to  match  network,  system,  and 
cognitive  performance  metrics  to  observed  behavior. 
This  triangulation  of  data  analysis  methods  provided  a 
quantification  of  human  performance  in  the  context  of 
network  and  system  performance.  These  insights  are 
valuable  as  mobile  ad  hoc  networks  become  available 
for  tactical  units.  These  networks  are  designed  to  be 
accommodating  to  network  disruptions  and  will  have 
self-healing  capabilities.  Engineering  these  networks  to 
optimize  human  performance  in  decision-making  areas 
has  been  shown  in  this  experiment  to  be  possible  under 
less  than  100  percent  bandwidth  conditions.  That  the 
soldiers  could  perform  well  in  significantly  degraded 
video  conditions  suggests  that  limited  bandwidth 
can  accommodate  decision  making  for  certain  task 
scenarios. 

The  availability  of  a  technology  such  as  GEC20  is 
the  critical  underpinning  of  this  analysis  activity.  This 
technology  allowed  after  action  playback/play-forward 
of  mission  runs  with  archival  review  of  messages,  spot 
reports,  and  sensor  images.  Thus,  the  analysis  team  was 
able  to  retrieve  SALUTE  reports  and  objectively 
contrast  the  content  of  each  message  with  actual 
instrumented  activity.  This  capability  significantly 
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reduced  the  need  to  record  detailed  activity  notes  on 
the  part  of  observers  and  solved  the  problem  inherent 
with  field  experiments  of  this  type  and  duration: 
remembering  and  distinguishing  between  events  of  one 
day  and  another. 

The  final  contribution  of  this  study  was  to  document 
the  ability  of  soldiers  to  conduct  planning,  execution, 
and  analysis  tasks  in  ISR  scenarios  with  a  range  of 
unmanned  systems,  communications  technologies,  and 
battle  command  systems.  That  these  disparate  battle 
command  systems  could  be  engineered  in  a  mobile  ad 
hoc  network  was  a  feat  of  merit  in  itself;  the  additional 
fact  that  the  soldiers  could  adaptively  work  between 
these  systems  was  of  particular  interest.  The  soldiers 
combined  their  experience  with  personal  and  Army 
computers  with  a  range  of  classroom  and  field  training 
in  preparation  for  the  field  experiment  and  performed 
with  low  workload  and  high  situational  awareness 
against  an  adaptive  enemy  force. 

This  analysis  investigated  the  strength  of  the 
theoretical  underpinnings  of  the  UBC  vision.  The 
methodological  approach  of  an  integrated  analysis  of  a 
system-of-systems  MANET  represents  a  pioneering 
first  step  toward  the  goal  of  network  optimization  for 
human  understanding  in  a  mission-relevant  environ¬ 
ment.  This  method  showed  promise  in  mapping 
performance  metrics  on  behavior  across  the  network 
levels.  Future  efforts  will  focus  on  the  continued 
resolution  of  quantifiable  quality  of  service  metrics, 
system  performance  characteristics,  and  human  deci¬ 
sion  making.  □ 
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Endnotes 

Network-Centric  Warfare  is  also  commonly  referred  to  as  Network¬ 
centric  operations  or  Network-enabled  operations.  For  ease  of  discussion, 
we  will  use  the  NCW  term  in  this  article  to  generically  refer  to  the 
concept  that  the  network  is  providing  information  communications 
technologies  to  Warfighters. 

2Forces  use  the  FBCB2  Blue  Force  Tracker  capability  to  navigate  when 
mobile.  A  1-minute  COP  update  rate  would  have  the  following  impact:  a 
vehicle  traveling  20  mph  would  cover  .5  miles  in  1  minute.  This  could  be 
an  acceptable  update  rate  or  not,  depending  upon  the  nature  of  the 
navigated  terrain.  Such  a  rate  would  clearly  be  inadequate  for  city  or 
village  navigation. 

3CERDEC  enhancements  to  the  FBCB2  display  included  windows  to 
provide  chat,  sensor  image  links,  and  a  view  of  network  health  status  of  all 
vehicles  in  the  network.  These  enhancements  were  made  by  linking 
additional  programs  to  the  display. 

4GEC20  was  developed  by  Mechdyne,  Future  Skys,  and  JB 
Management  at  the  direction  of  the  U.S.  Army  CERDEC  C2 
Directorate.  GEC20  was  used  in  this  study  as  a  tool  for  replaying  the 
missions  in  order  to  effectively  understand  the  sequence  and  timing  of 
events,  which  assisted  in  scoring  SA  measures. 
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